Stream processing typically follows the pattern of continuous queries, which may be thought of in some instances as being queries that execute for a potentially indefinite amount of time on data that can change very rapidly. Such rapidly changing data are called streams, and streams oftentimes comprise events. Such streams often exist in real-world scenarios, e.g., as temperature readings from sensors placed in warehouses or on trucks, weather data, entrance control systems (where events are generated whenever a person enters or leaves, for instance), etc. Events may include attributes (also sometimes referred to as a payload) such as, for example, the value of temperature readings and metadata (sometimes referred to as a header or header data) such as, for example, creation date, validity period, and quality of the event. Possible events occurring in an environment typically are schematically described by so-called event types, which in some respects are somewhat comparable to table definitions in relational databases. Streams may in certain scenarios be organized in channels that in turn are implemented by an event bus. Channels and event types in this sense may be considered orthogonal concepts, e.g., in the sense that channels may comprise events of several event types, and events of the same event type might be communicated via different channels.
Events in corresponding data streams are typically used in computer systems adhering to the event-driven architecture (EDA) paradigm. In such systems, several computer applications each execute on distinct computer systems and are typically interconnected by a network, such as a local area network or even the Internet. Each application typically is in charge of executing a certain processing task, which represents a processing step in an overall process, and each application typically communicates with the other applications by exchanging events. Examples include the calculation of complex mathematical models (e.g., for weather forecasts or scientific computations) by a plurality of distributed computers, the control of an assembly line (e.g. for the manufacturing of a vehicle, wherein each assembly step is controlled by a particular application participating in the overall assembly process), etc. Generally, events may be represented in a variety of different formats. The XML format, for instance, is one common format in which events may be represented. For example, an event stating the current conditions of a machine by giving values for temperature and oil pressure could be represented in the following manner:
<MachineState><header valid-from=”20120511:10:59:00”valid-to=”20120511:11:00:00”/><payload><temperature>165</temperature><oilpressure>7</oilpressure><powerconsumption>300</powerconsumption><state>running</state></payload></MachineState>
In a complex event processing (CEP) system, events may be evaluated and aggregated to form derived (or complex) events by so-called event processing agents. A typical manner to specify such evaluation and aggregation involves using CEP queries, which oftentimes are formulated in an SQL-like query language that is enhanced by some CEP-specific clauses such as, for example, a WINDOWS or RANGE clause to define conditions that relate to the occurrence of events within streams or channels. Typically, CEP systems are used to automatically trigger some activity, e.g., an appropriate reaction on an unusual situation that is reflected by the occurrence of some event patterns. The execution of such a reaction, however, typically lies outside of the CEP system. A common mechanism to trigger reactions includes having some agents listening for specific complex events on dedicated channels and executing the appropriate action when such an event is encountered.
FIG. 1 is a simplified schematic view of a typical CEP system and its environments. As shown in FIG. 1, an event bus 102 listens for events from, and publishes events to, one or more event processing agents 104, e.g., via one or more channels 106 provided to the event bus 102. Plural event processing agents 104a-n are listening for, and publishing events to, channels 106a-c in the FIG. 1 example. One or more agents 108 for executing actions also may listen to events from the event bus 102 via a suitable first channel 106a. Of course, it will be appreciated that the event bus 102 may have any number of channels 106 that listen for events from and/or publish events to any number of event processing agents 104, and/or may have any number of channels 106 that publish events to any number of agents for executing actions 108, in different implementations.
In contrast with database systems that run queries once a certain state of the data has been reached, CEP systems perform “continuous” query execution on streams, e.g., a query is “constantly” and “continuously” evaluated “forever.” This approach allows CEP systems to spend much more effort on query optimization, as query compilation typically occurs only once, unless the query is modified. On the other hand, CEP systems could benefit from a mechanism for “hot redeployment” of queries to cope with changes in queries.
It is believed that in current applications using CEP systems, the application semantics (which may be thought of in some cases as the logic underlying a particular application, such as whether a given sequence of processing steps is considered correct or incorrect with respect to the intended application behavior) is coded into the CEP queries. For example, in an application area pertaining to production systems, CEP can be used to monitor the production and detect situations that need human attention or even automatic reactions. In this case, all situations that are considered critical would be expressed by corresponding expressions in the CEP queries. When the related situation is detected, such a query would generate an event that triggers an adequate reaction. For example, if a machine reports a blockage, the whole production chain should be stopped. A sample query to handle this could be the following:
SELECT ‘StopProduction’FROM MachineState ON CHANNEL MachineConditionWHERE state=’blocked’;
This query assumes that a blockage is reported on channel MachineCondition and with an event type MachineState. If other channels exist that transport machine blockage events, these would be missed by the query. Also, machine blockages reported with a different event type would be missed.
Of course, there might be other conditions that should cause stop of production, e.g., gas leakages, etc. As a consequence, there might be another CEP Query such as:
SELECT ‘StopProduction’FROM SensorData ON CHANNEL EnvironmentWHERE gas>10;
Both queries could potentially be combined into a single one, e.g., as follows:
SELECT ‘StopProduction’FROM MachineState ON CHANNEL MachineConditionWHERE state=’blocked’UNIONSELECT ‘StopProduction’FROM SensorData ON CHANNEL EnvironmentWHERE gas>10;
There is, however, no means other than programming discipline to guarantee that the queries are combined in such a way that the combination represents a meaningful semantics. For example, there is nothing that prevents combinations of production stop conditions with other conditions, such as in the following query:
SELECT ‘StopProduction’FROM MachineState ON CHANNEL MachineConditionWHERE state=’blocked’ UNIONSELECT ‘StartVentilation’FROM SensorData ON CHANNEL EnvironmentWHERE gas>0;
Similar comments apply with respect to techniques for prevents typing errors (that may, for example, result in effectively missing a critical situation), e.g., as shown in the following query:
SELECT ‘StopProduction’FROM MachineState ON CHANNEL MachineConditionWHERE state=’blocked’UNIONSELECT ‘SopProduction’FROM SensorData ON CHANNEL EnvironmentWHERE gas>10;
When more of these and/or other critical situations are to be monitored, relevant states are reported via an additional channel, additional event types are used to report relevant states, and/or the like, the queries generally have to be enhanced or modified. As a consequence, application semantics is scattered over many queries. It accordingly is hard to keep an overview of the semantics of the application. Furthermore, there is no reasonable means for verifying that all the necessary changes have been done (e.g., adding another channel might require changes in many queries) and whether they have been done correctly (e.g., spelling errors as shown before). This is particularly true when queries are more complex, e.g., as in the following query that triggers a maintenance request for a certain machine when the number of warning conditions (temperature too high, oil pressure low, system reset) within an hour is above a certain threshold (here, 5):
SELECT ‘RequestMaintenance’, machine FROM(SELECT COUNT(a.x) AS number, machineFROM (SELECT machineFROM MachineState ON CHANNEL MachineConditionWHERE temperature > 100 or oilpressure < 10UNIONSELECT machineFROM MachineEvents ON CHANNEL ProductionEventsWHERE eventtype=’SystemReset’)WINDOW(RANGE 1 HOUR)GROUP BY machine)WHERE number > 5
The inventor has realized that one way to address such issues would involve employing an ontology in connection with CEP systems and/or methods. Ontologies are commonly used ways of modeling parts of the world by using semantic concepts. It is noted that some CEP systems have been based on taxonomies. Ontologies, however, are much more expressive than taxonomies, which at best only provide a strict hierarchical order of terms without any further semantics.
WO 2012/034187, which is hereby incorporated by reference herein in its entirety, describes a mechanism for defining CEP event types on the basis of an ontology such that, with suitable transformation specific to a CEP engine, the event processing can eventually be based on the ontology definition. This approach in some senses attempts to combine an ontology with CEP processing, relying on the ontology being the driving force to process events based on the ontology by a specific transformation for each CEP engine.
In contrast with the approach taken in WO '187, certain example embodiments involve separating the semantics from queries and allowing fast, easy, and transparent extension and change of the semantics without necessarily forcing any changes in the operative queries, while on the other hand being able to use the full query language expressiveness supported by the CEP engine.
The approaches employed by certain example embodiments advantageously do not require the creation of the initial event types or queries by an ontology. Indeed, one cannot reasonably expect to cover each functionality of a concrete CEP language when the starting point is at the ontology from which the CEP query will be generated. In marked contrast, certain example embodiments start with the full extent of the CEP query language and do not restrict it. In addition, given the approach taken by WO '187, it does not seem possible to dynamically correlate single events (for example, also by a third party or humans) with an ontology class.
As will be appreciated from the above, then, each CEP system usually provides its own version of some sort of query language, and these query languages have their common drawbacks with respect to specifying semantic criteria in a dynamic way. Certain example embodiments, by contrast, enrich a CEP query language with the power of an ontology, e.g., to define the semantics in more powerful and dynamic ways.
One aspect of certain example embodiments relates to the separation of the semantics and the operative CEP language. Using an ontology increases the flexibility and advantageously results in richer and easier usage of CEP systems.
Another aspect of certain example embodiments relates to the fact that because the end-result of the extension is not visible to the CEP engine, the usage may be independent of any particular CEP language. Instead, a pre-compile step may be provided to enable this further division.
In certain example embodiments, an event processing system is provided. An event bus is configured to receive a stream of events, with each said event having a predefined event type. An event processing agent includes at least one first processor and a memory. The event processing agent is configured to execute predefined queries on the events, with each said query conforming to a query language, and with the query language being enhanced via a semantic extension corresponding to an ontology. An ontology management component is in communication with the event processing agent. The ontology management includes a first storage location storing mappings (optionally specified at design time) between CEP concepts (e.g. events, channels, etc.) and concepts of the ontology (e.g. ontology classes) that enhances the query language; and processing resources, including at least one second processor and a memory, configured to translate (optionally at compile time) references to ontology concepts into translated queries processable by the event processing agent in accordance with the query language, without the semantic extension enhancement. In other words, in certain example embodiments, the mapping may map event types to ontology classes and, rather than translating ontology rules, references to ontology concepts embedded in the query language may be translated.
According to certain example embodiments, CEP concepts may include events, event types, and channels, and/or concepts of the ontology may include classes. Further, according to certain example embodiments, attributes of event types may be mappable to properties of classes. Still further, according to certain example embodiments, translated queries may be free from, and lack any reference to, any ontology concepts.
In certain example embodiments, an event processing agent is provided. Processing resources include at least one first processor and a memory. A first connection is provided to a channel on an event bus configured to receive a stream of events. Each said event has a predefined event type. A second connection is provided to an ontology management component. The event processing agent, in cooperation with the processing resources, is configured to execute predefined queries on events received over the first connection. Each said query conforms to a query language, and the query language is enhanced via a semantic extension corresponding to an ontology. The ontology management includes a first storage location storing mappings between concepts of the query language and concepts of the ontology that enhances the query language, and is configured to translate references to ontology concepts into translated queries processable by the event processing agent in accordance with the query language, without the semantic extension enhancement.
In certain example embodiments, a method of processing events in a complex event processing (CEP) system is provided. A stream of events is received via an event bus, with each said event having a predefined event type. It is determined, in connection with an event processing agent including at least one processor, when a particular query out of a plurality of possible queries is to be executed on an event. Each said query either initially (e.g., as defined) conforms to a CEP query language executable by the event processing agent and thus is executable by the event processing agent, or has been translated into a translated query executable via the event processing agent from an enhanced query that conforms to a version of the CEP query language that has been enriched so that semantics thereof are represented in accordance with an ontology. The particular query is executed in connection with the at least one processor of the event processing agent. In some cases, translated queries may be free from, and lack any reference to, any ontology concepts.
In certain example embodiments, there is provided a method of configuring a complex event processing (CEP) system in which a stream of events is received via an event bus. Each said event has a predefined event type. Queries that are to be executable in connection with the CEP system are received. It is determined, in connection with at least one processor, whether a received query either initially conforms to a CEP query language executable by an event processing agent, or must be translated from an enhanced query that conforms to a version of the CEP query language that has been enriched so that semantics thereof are represented in accordance with an ontology in order to render it executable via the event processing agent. When the received query must be translated, a translated query is generated from the enhanced query in accordance with mappings between concepts of the CEP system and concepts of the ontology. All queries that initially conform to the CEP query language and all translated queries are deployed for possible subsequent execution.
In certain example embodiments, there is provided a non-transitory computer readable storage medium tangibly storing instructions that, when executed by at least one processor of a system, perform a method as described herein.
These aspects, features, and example embodiments may be used separately and/or applied in various combinations to achieve yet further embodiments of this invention.